CLAIMS 



What is claimed is: 

1. An apparatus for creating, managing and publishing interactive virtual tours, the 
apparatus comprising: 

a panorama data acquisition unit implementing means of capturing panoramic 
data and optionally preparing said panoramic data for further processing; 

a transform engine responsive to said panoramic data and implementing 
means of correcting distortions in said panoramic data and/or performing 
automatic, manual or interactive calibration of said panoramic data and/or 
transforming said panoramic data into a desired format or set of formats when 
necessary, wherein said distortion correction could optionally be 
accomplished through steps of: 

a) loading data including reference data that could be used to derive 
panoramic imaging system distortion profiles; 

b) specifying a linear or other pre-determined distortion function or set of 
distortion functions and optionally displaying representations of said 
distortion function or set of distortion functions; 

c) using distortion function or set of distortion functions to build a 
distortion profile for the panoramic imaging system that was used to 
acquire the reference data; 

d) performing transformation on panoramic data using distortion profile 
specified in c) and optionally displaying results of said transformation; 
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e) determining whether distortion is satisfactorily corrected; 

f) continuing with steps g) and h) if distortion is deemed not to have been 
satisfactorily corrected and continuing with step i) if distortion is 
deemed to have been satisfactorily corrected; 

g) automatically or interactively modifying the distortion function or set 
of distortion functions using feedback from optionally displayed 
distortion function representation or transformation results obtained in 
step d) or by using data loaded in step a); 

h) repeating steps c) to g) until distortion is judged to have been 
satisfactorily corrected; 

i) storing distortion profile obtained in step c) for use in performing 
transformations on panoramic data acquired using the panoramic 
imaging system for which data was loaded in step a); 



an optional package generator adapted to generate virtual tour packages 
containing said panoramic data, commands and/or optional virtual tour data; 

a viewing engine responsive to said panoramic data and/or virtual tour 
packages and implementing means for perspective correction, and user 
interaction with, said panoramic data and/or optional virtual tour data when 
necessary; 

an optional control engine adapted to fecilitate a higher level of interaction 
with said panoramic data and/or optional virtual tour data, wherein said 
control engine is optionally connected operatively to and communicates bi- 
directionally with said viewing engine, renders representative information 
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about all or parts of said virtual tour, permits a particular portion to be 
selected from said virtual tour and optionally sends signals to said viewing 
engine that cause said viewing engine to permit the interactive navigation of 
said selected portion of said virtual tour, wherein said control engine also 
indicates or causes to be indicated what portion of said virtual tour is currently 
selected and what sub-part of said selected portion of said virtual tour is 
currently rendered, wherein said control engine is responsive to user input 
and/or commands from said viewing engine and is in turn capable of 
modifying said representative information about all or parts of said virtual 
tour in response to said user input and/or said commands from said viewing 
engine and is further optionally capable of communicating information 
indicative of such externally induced modifications to said user and/or said 
viewing engine; 

a display means for rendering output of said viewing engine, control engine, 
package generator, transform engine, and/or panoramic data acquisition unit. 

The apparatus of claim 1, wherein said panoramic data acquisition unit is adapted 
to capture data representing 2D panoramic images or video, 3D or stereoscopic 
panoramic images or video, holographic data, infrared data, ultrasonic data or 
ultraviolet data. 

The apparatus of claim 1 , wherein said panoramic data acquisition unit comprises 
at least one versatile device for creating representations of stimuli covering 
substantially all directions around a given reference or view point, said versatile 
device for creating representations of stimuli comprising at least one grid of one 
or more focusing elements disposed on an N-dimensional and arbitrarily shaped 
surface, at least one grid of one or more sensor elements disposed on an N- 
dimensional and arbitrarily shaped surface, and optionally, at least one grid of one 
or more stimulus guide elements disposed on an N-dimensional and arbitrarily 
shaped surfece, wherein said focusing element grid is adapted to focus stimuli 
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covering substantially all directions around a given reference or view point onto 
the sensor element grid and may optionally do so via the stimulus guide unit if 
necessary, wherein each focusing element or group of focusing elements is 
associated with and focuses a subset (typically the subset impinging on it) of the 
entire stimulus space onto a sensor element or group of sensor elements 
responsive to the stimuli, wherein when an optional stimulus guide element grid is 
provided, the focusing element grid can be adapted to focus the stimuli onto the 
stimulus guide element grid for suitable formatting and onward transmission to 
the sensor element grid, wherein when used, each stimulus guide element or 
group of elements is associated with and receives stimuli from a focusing element 
or group of focusing elements and is in turn associated with and transmits stimuli 
to a sensor element or group of sensor elements. 

4. The apparatus of claim 3, wherein said versatile device for creating 
representations of stimuli covering substantially all directions around a given 
reference or view point comprises at least one grid of one or more sensor elements 
disposed on an N-dimensional and arbitrarily shaped surface, said at least one grid 
of one or more sensor elements responsive to electromagnetic radiation. 

5. The apparatus of claim 1 , wherein said panoramic data acquisition unit comprises 
a spherical image/video acquisition unit comprising at least one grid of one or 
more photosensitive elements on a surface with a spherical geometry or an 
approximation thereto and at least one enclosing concentric grid of one or more 
focusing elements on a surface with a spherical geometry or an approximation 
thereto. 

6. The apparatus of claim 1 , wherein said panoramic data acquisition unit comprises 
a panoramic imaging system characterized by a 360-degree lateral field of view 
and a vertical field of view that is usually less than 1 80 degrees. 
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7. The apparatus of claim 1 , wherein said panoramic data acquisition unit is adapted 
to capture each complete panoramic data block in a single image/video frame. 

8. The apparatus of claim 1 , wherein said panoramic data acquisition unit is adapted 
to capture each complete panoramic data block in a plurality of image/video 
frames that are combined to form complete panoramic data blocks. 

9. The apparatus of claim 1, wherein said associated optional virtual tour data 
includes audio, video, text, graphics, pointers to resources on local and distributed 
networks or virtual tour packages. 

10. The apparatus of claim 1, wherein said panoramic data acquisition unit further 
comprises means of preparing and transferring all or portions of acquired 
panoramic data to said transform engine, said optional package generator, said 
viewing engine, said optional control engine or said display; wherein said 
preparation and/or transfer optionally involves compression and/or decompression 
of all or portions of acquired panoramic data; wherein said preparation and/or 
transfer is optionally in response to user commands or information inferred from 
the state of said viewing engine or said control engine. 

11. The apparatus of claim 1, wherein said transform engine implements means of 
converting panoramic data from a first format to one or more formats and/or vice 
versa. 

12. The apparatus of claim 11, wherein said panoramic data conversion is between 
polar and rectilinear coordinates, polar and spherical coordinates, spherical and 
rectilinear coordinates, cubic and spherical coordinates, cubic and polar 
coordinates or cubic and rectilinear coordinates. 

13. The apparatus of claim 1, wherein said distortion function or set of distortion 
functions is based on a polynomial or a set of polynomials of suitable degree. 
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14. The apparatus of claim 1, wherein said package generator implements means of 
specifying active regions and/or volumes on said panoramic data and/or virtual 
tour data for programming reference. 

15. The apparatus of claim 1, wherein said package generator implements means of 
specifying navigable paths or walk-through sequences on said panoramic data 
and/or virtual tour data. 

16. The apparatus of claim 15, wherein said navigable paths or walk-through 
sequences are navigated in automatic or guided mode. 

17. The apparatus of claim 1, wherein said panoramic data, virtual tour data, 
commands, navigable paths or walk-through sequences, virtual tour packages 
and/or distortion profiles are managed using a universal file format, said universal 
file format specifying a header identifying the file type and containing 
information as to the number, types, locations and sizes of elements it contains, 
wherein each element in the file is in turn described by a header specifying the 
type of element, its size and any relevant data, commands or attributes and the 
types, locations and sizes of any additional elements it contains, thus making it 
possible for arbitrary types and numbers of elements to be managed by said 
universal file format 

18. The apparatus of claim 1, wherein said viewing engine is responsive to user input 
and implements means for perspective correction of said panoramic data, 
navigable paths or walk-through sequences and/or virtual tour packages. 

19. The apparatus of claim 1, wherein said viewing engine further implements means 
of navigating said panoramic data, navigable paths or walk-through sequences 
and/or virtual tour packages. 
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20. The apparatus of claim 1, wherein said viewing engine implements means for a 
single user or a plurality of users to independently and simultaneously interact 
with and navigate said panoramic data, navigable paths or walk-through 
sequences and/or virtual tour packages. 

21. The apparatus of claim 1, wherein said viewing engine implements means for a 
plurality of users to interact with and navigate said panoramic data, navigable 
paths or walk-through sequences and/or virtual tour packages in a collaborative, 
competitive, coordinated, and/or synchronized manner. 

22. The apparatus of claim 1, wherein said representative information rendered by 
said control engine about all or parts of said virtual tour is rendered in 2D, 3D or 
higher dimensional space and/or time. 

23. The apparatus of claim 1, wherein said viewing engine and control engine 
comprise a unitary structure. 

24. A method of creating, managing and publishing interactive virtual tours, said 
method comprising: 

a panorama data acquisition step for capturing panoramic data and optionally 
preparing said panoramic data for further processing; 

a transform step for correcting distortions in said panoramic data and/or 
performing automatic, manual or interactive calibration of said panoramic 
data and/or transforming said panoramic data into a desired format or set of 
formats when necessary, wherein said distortion correction could optionally 
be accomplished through steps of: 

a) loading data including reference data that could be used to derive 
panoramic imaging system distortion profiles; 
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b) specifying a linear or other pre-determined distortion function or set of 
distortion functions and optionally displaying representations of said 
distortion function or set of distortion functions; 

c) using distortion function or set of distortion functions to build a 
distortion profile for the panoramic imaging system that was used to 
acquire the reference data; 

d) performing transformation on panoramic data using distortion profile 
specified in c) and optionally displaying results of said transformation; 

e) determining whether distortion is satisfactorily corrected; 

f) continuing with steps g) and h) if distortion is deemed not to have been 
satisfactorily corrected and continuing with step i) if distortion is 
deemed to have been satisfactorily corrected; 

g) automatically or interactively modifying the distortion function or set 
of distortion functions using feedback from optionally displayed 
distortion function representation or transformation results obtained in 
step d) or by using data loaded in step a); 

h) repeating steps c) to g) until distortion is judged to have been 
satisfactorily corrected; 

i) storing distortion profile obtained in step c) for use in performing 
transformations on panoramic data acquired using the panoramic 
imaging system for which data was loaded in step a); 
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an optional package generation step adapted to generate virtual tour packages 
containing said panoramic data, commands and/or optional virtual tour data; 

a viewing step responsive to said panoramic data and/or virtual tour packages 
and providing means for perspective correction of, and user interaction with, 
said panoramic data and/or optional virtual tour data when necessary; 

an optional control step adapted to facilitate a higher level of interaction with 
said panoramic data and/or optional virtual tour data, wherein said control step 
is optionally connected operatively to and communicates bi-directionally with 
said viewing step, renders representative information about all or parts of said 
virtual tour, permits a particular portion to be selected from said virtual tour 
and optionally sends signals to said viewing step that cause said viewing step 
to permit the interactive navigation of said selected portion of said virtual 
tour, wherein said control step also indicates or causes to be indicated what 
portion of said virtual tour is currently selected and what sub-part of said 
selected portion of said virtual tour is currently rendered, wherein said control 
step is responsive to user input and/or commands from said viewing step and 
is in turn capable of modifying said representative information about all or 
parts of said virtual tour in response to said user input and/or said commands 
from said viewing step and is further optionally capable of communicating 
information indicative of such externally induced modifications to said user 
and/or said viewing step; 

a display step providing means for rendering output of said viewing step, 
control step, package generation step, transform step, and/or panoramic data 
acquisition step. 

25. The method of claim 24, wherein said panoramic data acquisition step is adapted 
to capture data representing 2D panoramic images or video, 3D or stereoscopic 



30 



panoramic images or video, holographic data, infrared data, ultrasonic data or 
ultraviolet data 

26. The method of claim 24, wherein said panoramic data acquisition step utilizes at 
least one versatile device for creating representations of stimuli covering 
substantially all directions around a given reference or view point, said versatile 
device for creating representations of stimuli comprising at least one grid of one 
or more focusing elements disposed on an N-dimensional and arbitrarily shaped 
surface, at least one grid of one or more sensor elements disposed on an N- 
dimensional and arbitrarily shaped surface, and optionally, at least one grid of one 
or more stimulus guide elements disposed on an N-dimensional and arbitrarily 
shaped surface, wherein said focusing element grid is adapted to focus stimuli 
covering substantially all directions around a given reference or view point onto 
the sensor element grid and may optionally do so via the stimulus guide unit if 
necessary, wherein each focusing element or group of focusing elements is 
associated with and focuses a subset (typically the subset impinging on it) of the 
entire stimulus space onto a sensor element or group of sensor elements 
responsive to the stimuli, wherein when an optional stimulus guide element grid is 
provided, the focusing element grid can be adapted to focus the stimuli onto the 
stimulus guide element grid for suitable formatting and onward transmission to 
the sensor element grid, wherein when used, each stimulus guide element or 
group of elements is associated with and receives stimuli from a focusing element 
or group of focusing elements and is in turn associated with and transmits stimuli 
to a sensor element or group of sensor elements. 

27. The method of claim 26, wherein said versatile device for creating representations 
of stimuli covering substantially all directions around a given reference or view 
point comprises at least one grid of one or more sensor elements disposed on an 
N-dimensional and arbitrarily shaped surface, said at least one grid of one or more 
sensor elements responsive to electromagnetic radiation. 
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28. The method of claim 24, wherein said panoramic data acquisition step utilizes a 
spherical image/video acquisition unit comprising at least one grid of one or more 
photosensitive elements on a surface with a spherical geometry or an 
approximation thereto and at least one enclosing concentric grid of one or more 
focusing elements on a surface with a spherical geometry or an approximation 
thereto. 

29. The method of claim 24, wherein said panoramic data acquisition step utilizes a 
panoramic imaging system characterized by a 360-degree lateral field of view and 
a vertical field of view that is usually less than 180 degrees. 

30. The method of claim 24, wherein said panoramic data acquisition step is adapted 
to capture each complete panoramic data block in a single image/video frame. 

31 . The method of claim 24, wherein said panoramic data acquisition step is adapted 
to capture each complete panoramic data block in a plurality of image/video 
frames that are combined to form complete panoramic data blocks. 

32. The method of claim 24, wherein said optional virtual tour data includes audio, 
video, text, graphics, pointers to resources on local and distributed networks or 
virtual tour packages. 

33. The method of claim 24, wherein said panoramic data acquisition step further 
comprises providing means of preparing and transferring all or portions of 
acquired panoramic data to said transform engine, said optional package 
generator, said viewing engine, said optional control engine or said display; 
wherein said preparation and/or transfer optionally involves compression and/or 
decompression of all or portions of acquired panoramic data; wherein said 
preparation and/or transfer is optionally in response to user commands or 
information inferred from the state of said viewing engine or said control engine. 
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34. The method of claim 24, wherein said transform step implements means of 
converting panoramic data from a first format to one or more formats and/or vice 
versa. 

35. The method of claim 34, wherein said panoramic data conversion is between 
polar and rectilinear coordinates, polar and spherical coordinates, spherical and 
rectilinear coordinates, cubic and spherical coordinates, cubic and polar 
coordinates or cubic and rectilinear coordinates. 

36. The method of claim 24, wherein said distortion function or set of distortion 
functions is based on a polynomial or a set of polynomials of suitable degree. 

37. The method of claim 24, wherein said package generation step implements means 
of specifying active regions and/or volumes on said panoramic data and/or virtual 
tour data for programming reference. 

38. The method of claim 24, wherein said package generation step implements means 
of specifying navigable paths or walk-through sequences on said panoramic data 
and/or virtual tour data. 

39. The method of claim 38, wherein said navigable paths or walk-through sequences 
are navigated in automatic or guided mode. 

40. The method of claim 24, wherein said panoramic data, virtual tour data, 
commands, navigable paths or walk-through sequences, virtual tour packages 
and/or distortion profiles are managed using a universal file format, said universal 
file format specifying a header identifying the file type and containing 
information as to the number, types, locations and sizes of elements it contains, 
wherein each element in the file is in turn described by a header specifying the 
type of element, its size and any relevant data, commands or attributes and the 
types, locations and sizes of any additional elements it contains, thus making it 
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possible for arbitrary types and numbers of elements to be managed by said 
universal file format 



41 . The method of claim 26, wherein said viewing step is responsive to user input and 
implements means for perspective correction of said panoramic data, navigable 
paths or walk-through sequences and/or virtual tour packages. 

42. The method of claim 24, wherein said viewing step further implements means of 
navigating said panoramic data, navigable paths or walk-through sequences 
and/or virtual tour packages. 

43. The method of claim 24, wherein said viewing step implements means for a single 
user or a plurality of users to independently and simultaneously interact with and 
navigate said panoramic data, navigable paths or walk-through sequences and/or 
virtual tour packages. 

44. The method of claim 24, wherein said viewing step implements means for a 
plurality of users to interact with and navigate said panoramic data, navigable 
paths or walk-through sequences and/or virtual tour packages in a collaborative, 
competitive, coordinated, and/or synchronized manner. 

45. The method of claim 24, wherein said representative information rendered by said 
control step about all or parts of said virtual tour is rendered in 2D, 3D or higher 
dimensional space and/or time. 

46. The method of claim 26, wherein said viewing step and control step are 
implemented in a unitary structure. 
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